This is where pathology lives. This sort of thinking is fundamental to every test that we perform, every diagnosis that we render. A few sticking points that I (and most people) struggle with:
The essentials of sensitivity and specificity are made memorable by SPIN (specific to rule in) and SNOUT (sensitive to rule out).
| Test Result | Disease | No Disease | Totals |
|---|---|---|---|
| Positive | A | B | A + B |
| Negative | C | D | C + D |
| Totals | A + C | B + D | A + B + C + D |
\[Accuracy = (A+B)/(A+B+C+D)\] \[Sensitivity=A/(A+C)\] \[Specificity=D/(B+D)\] \[Positive Predictive Value=A/(A+B)\] \[Negative Predictive Value=D/(C+D)\]
Let’s see, for instance, how prevalence affects the behavior of a test. Make the following assumptions: Prevalence=0.01 (1%) Sensitivity of our test=90% Specificity of our test=90% We have 1000 patients to examine.
| Test Result | Disease | No Disease | Totals |
|---|---|---|---|
| Positive | 9 | 99 | 108 |
| Negative | 1 | 891 | 892 |
| Totals | 10 | 990 | 1000 |
\(Positive Predictive Value = A/(A+B) = 9/108 = 8.3\)%
\(Negative Predictive Value = D/(C+D) = 891/892 = 99\)%
Now, when prevalence changes, our PPV and NPV also change: Prevalence=0.1 (10%) Sensitivity of our test=90% Specificity of our test=90% We have 1000 patients to examine.
| Test Result | Disease | No Disease | Totals |
|---|---|---|---|
| Positive | 90 | 90 | 180 |
| Negative | 10 | 810 | 820 |
| Totals | 100 | 900 | 1000 |
\(Positive Predictive Value=A/(A+B)=90/180=50\)%
\(Negative Predictive Value=D/(C+D)=810/820=99\)%
Prevalence=0.5 (50%) Sensitivity of our test=90% Specificity of our test=90% We have 1000 patients to examine.
| Test Result | Disease | No Disease | Totals |
|---|---|---|---|
| Positive | 450 | 50 | 500 |
| Negative | 50 | 450 | 500 |
| Totals | 500 | 500 | 1000 |
\(Positive Predictive Value=A/(A+B)=450/500=90\)%
\(Negative Predictive Value=D/(C+D)=450/500=90\)%
Prevalence=0.9 (90%) Sensitivity of our test=90% Specificity of our test=90% We have 1000 patients to examine.
| Test Result | Disease | No Disease | Totals |
|---|---|---|---|
| Positive | 810 | 10 | 820 |
| Negative | 90 | 90 | 180 |
| Totals | 900 | 100 | 1000 |
\(Positive Predictive Value=A/(A+B)=810/820=99\)% \(Negative Predictive Value=D/(C+D)=90/180=50\)%
And remember, this performance is from the same test.
So when it comes to rare (or even uncommon) disease and immunohistochemical stains that are often less than 90% sensitive and 90% specific, the negative predictive value outperforms the positive predictive value of the tests. Conversely, when the pre-test probability of something is very high, the positive predictive value of the test outweighs the negative predictive value. So adding an immunohistochemical stain to a case when you’re 99% sure it’s disease X and you’re looking for positive confirmation is not only wasteful, it’s stupid. Instead, focus on useful ruleouts or stay put with what you’ve got.
By useful ruleouts, I mean diseases which entail a different treatment, a different prognosis and which have relatively specific antibodies or characteristic immunohistochemical staining patterns which may lend additional specificity. This half of anatomic pathology is poorly understood by most clinicians and some pathologists, who should know better. Problems arise when pathologists are ignorant of these principles. Problems also arise when the definitions of diseases are poorly worked out, when the literature doesn’t focus on high quality diagnostic thinking, and when published sensitivity, specificity and prevalence are not available. Experts see a high prevalence of rare disease, so must understand the context of every case (consult, non-consult, quality of the material, and clinical setting including age, sex and imaging).
Notice that we haven’t even mentioned one of the most important topics in testing: the gold standard. This theme has never been more important, what with the proliferation of laboratory and imaging tests. We tend to look down upon older, more familiar tests and favor newer tests that at first glance offer better sensitivity and specificity. Trouble is, most new tests are not accompanied by rigorous outcomes studies (our real “gold standard,” if you will). A new test can never perform better than the old gold standard, unless additional information is taken into account. The reason for this is often unclear unless you spend your days thinking about testing and making diagnoses. But simply ask yourself “How do I know that a patient has disease X?” Clinicians base their clinical diagnoses on a complex constellation of factors that changes by patient (changing populations) and disease (changing characteristics, different availability of reliable ancillary testing or sensitivity/specificity of presenting signs and symptoms, which should be viewed as tests in their own right!)
Most of the time, laboratory testing or imaging studies are performed when there is a reasonable pre-test probability of disease (analogous to the patient being derived from a population having a high prevalence of the disease of concern). A highly sensitive, inexpensive test might be useful as a screen and a more specific test, if possible a “gold standard,” might be used to follow up. Usually if the patient is positive, we conclude that the patient has the disease.
However, real cases go undetected for two reasons:
If one proposes to replace an old gold standard, one has to come up with new information, since the gold standard is used to determine our diseased category. Radiology is especially guilty of hubris in this area, since they have complex tests which seem to (but in fact do not) sidestep the issues of sensitivity, specificity, PPV, NPV and gold standards. And the truly important information—how long will the patient live, whether he or she will respond to the proposed therapy, whether he or she is healthier or happier—is often not gathered.
Why? Because the output is complex, the diagnostic criteria poorly developed, or because follow up is too time-consuming or costly. The latter is particularly problematic, since emerging technologies have an incentive to make grand claims about their qualities, both for financial and academic reasons. And these diagnosticians earnestly want to help people. Combine equal parts profit motive with faith that your nifty new test can make a difference, along with ignorance of testing principles and you encounter precisely the situation that we find ourselves in today: inflated health care testing costs, a conflict of interest for diagnosticians, and a proliferation of tests that provide marginal utility.
OK. One new line of thinking has to be introduced at this point, and it may alter the above argument to a degree, and that is the notion of gene testing. This needs to be carved out in a separate discussion. Suffice it to say that some mutational analysis appears to be more sensitive and more specific than other tests that we routinely perform. Issues of sensitivity and specificity still play into the conversation, for example 1p/19q deletion is not specific for oligodendroglioma, and IDH1/2 mutation testing in the classic sense (i.e. PCR-based) relies on a certain amount of DNA with the mutation so that we can “dig signal out of the noise.” Nonetheless, certain germline mutations, for example seem to be 100% predictive of disease.
So what recommendations do I have for trainees?
Role of cytologic preparations in intra-operative diagnosis
Role of neuro-imaging in diagnostic neuropathology
Resources provided by this work: Text with high-quality images Posters with common problems YouTube lectures for each chapter (one or more per chapter) YouTube demonstrations of all procedures Adult brain autopsy Fetal brain autopsy Smear preparation and staining Aperio-scanned slides of all common tumors: smears and histologic slides
Techniques to get this manual written Visit key people, if they’ll let me, to see how they do business Rebecca Folkerth–fetal autopsy Joe Parisi–surgical neuropathology, dementia pathology, adult autopsy UCSF–surgical neuropathology with Arie Parry and Andy Bollen, catalog the brain collection there (scan?)
I’m not interested in answering many of the controversial questions described in this text: my task is much more realistic. I’m interested only in framing better questions or framing the usual questions in the best manner possible. When reasonable, I offer proposed pragmatic suggestions for resolving or circumventing these problems.
Statistical learning (machine learning) now seems to offer a pathway out of this mess. I need to incorporate thoughts from my PitAdTMA series of experiments to augment this.
summary(cars)
## speed dist
## Min. : 4.0 Min. : 2.00
## 1st Qu.:12.0 1st Qu.: 26.00
## Median :15.0 Median : 36.00
## Mean :15.4 Mean : 42.98
## 3rd Qu.:19.0 3rd Qu.: 56.00
## Max. :25.0 Max. :120.00
You can also embed plots, for example:
Note that the echo = FALSE parameter was added to the code chunk to prevent printing of the R code that generated the plot.
Make sure to comment out your code, but what role comments in RMD documents?
Hey, should I write the GATA3 paper in RMD? Small changes are not worth committing, since a committed document is the “gold standard,” the “display copy,” if you like. Nonetheless, the habit of committing is important.
Why use the git interface in RStudio rather than the terminal in RStudio? Doesn’t seem to add much, and it keeps me slightly further from git…
How do references work in Rmd? Here’s a reference dropped in with citr(1,2). I finally got citr installed when I realized that my R version was not the latest, and was preventing the loading of some dependent packages. Dumb, I know.
YAML specifications including output, bibliography, and CSL are important for inserting citations.
library(tidyverse)
ggplot(data = mpg) +
geom_point(mapping = aes(x = displ, y = hwy, color = class))
Why not make this into a page that contains useful things, like math? I should learn some LATEX(?), anyway.
Images can also be inserted as below.
Images will build up, so are placed in a folder that I set up in the Rproj folder.
1. D&D basic rules: Dungeon master’s basic rules version 0.5. 2018.
2. D&D basic rules: Player’s basic rules version 0.3. 2018.